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ABSTRACT 

This paper reports on the development of a new information communications technology (ICT) learning preference 
survey, its cross-validation with attitudes towards mobile learning, and new perspectives on information seeking, 
information sharing, and mobile access derived from the relationships uncovered. The Information and 
Communications Technology Learning (ICTL) survey instrument is designed to measure learners’ preferences and 
information behavior for learning, knowledge acquisition, and sharing in the Web 2.0 technology pervasive 
environments of the twenty-first century. The instrument development process includes analysis for reliability and 
validity of instrument scales. The multi-step refinement process revealed two constructs with, respectively, very good 
and respectable measurement properties: Information Sharing (alpha = .83) and Information Seeking (alpha = .71). 
Cross-validation with a newly established mobile learning instrument confirms that both the Information Sharing and 
Information Seeking scales of ICTL have significant (p < .001) alignment with mobile learning attitudes. Implications 
of findings are discussed. 
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1. INTRODUCTION 

Collaboration fostered by a doctoral level psychometrics course and scholarly interest in the role of mobile 
technologies in learning prompted the authors to develop and validate an instrument to measure learners’ 
dispositions toward new learning and knowledge acquisition activities that are made available by information 
and communications technology (ICT). The authors also sought to examine relationships between the ICT 
survey and an established mobile learning survey. 

The two surveys employed in this research study were: 

1) The Information and Connnunications Technology Learning (ICTL) survey, and 

2) The Mobile Learning Scale (MLS). 

The development of the Information and Communications Technology Learning survey is described in 
detail in subsequent sections of this paper. The Mobile Learning Scale vI.O also plays a key role in this study 
and therefore its development history and psychometric properties are briefly discussed. MLS vI.O was 
originally created from the key points developed for a paper on mobile learning prospects for informal 
learning in higher education (Khaddage & Knezek, 2011). Many of these key points initially emerged during 
working group discussions of the 2011 International Sunnnit on ICT in Education (UNESCO, Paris, 2011) 
(Knezek, Lai, Khaddage, & Baker, 2011). Explorations of the relationships among the emerging factors 
uncovered the relationships that are reflected in the title of this paper and are also addressed in details in the 
discussion. These findings illustrate that cross-validation techniques can contribute to the richness of 
understanding as gauged by the underlying constructs found in the cross-validation of instruments with 
established reliability. 
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2. CONCEPTUAL RATIONALE 

Measuring student technology attitudes related to learning and information seeking in technology pervasive 
learning environments of the twenty-first century is of particular interest to educators and learning 
technologists who seek to integrate ICT for effective teaching and learning. Also of great interest are 
learners’ attitudes towards learning with mobile technologies. The role of mobile technology in education is 
yet to be clearly defined, and therefore it is reasonable that many current papers present convincing 
arguments about ways that that mobile technologies and applications can bring many benefits to the learning 
environment. For example, Khaddage and Latterman (2012) point out that mobile technologies and 
application offer benefits such as portability, simplicity, and availability. A multi-year study conducted by 
Kolko and colleagues at the University of Washington reports on the adoption of ICT in Central Asia stating 
that “mobile phone usage is outpacing the rate of Internet adoption, that access to the Internet is primarily 
through public access sites”. Also reported is a trend towards the combination and conflation of technology 
related information seeking and communication usage patterns (Kolko, Rose & Johnson, 2007). 

The seemingly unlimited new media contexts and information access options in formal and informal 
settings are often regarded as venues of invaluable opportunity for teaching and learning in the twenty-first 
century (Arnone, Small, Chauncey & McKenna, 2011). There are those who warn that education that does 
not utilize these new media-based technologies will not adequately develop student potential, leaving 
students with a deficit in digital literacies that may contribute to student disengagement from classroom 
activities (Judson, 2010). Felt (2010) identifies new media literacy (NML) skills as a set of cultural 
competencies, social skills, and ICT tool skills that are essential for learning and mastery in the twenty-first 
century. New media literacy skills are being associated with content-based learning in the framework of 
Jenkins and colleagues (Jenkins, Clinton, Purushotma, Robison, Weigel, 2006), and are thought to support 
student thinking and learning. 

With information connnunications technology (ICT) now commonplace in all facets of our daily lives, 
including teaching and learning (Christensen & Knezek, 2006), it is important to understand student 
information seeking behavior in the Web 2.0 framework for formal and informal learning. Kuhlthau's 
Information Search Process (ISP) model, devised in the 1980’ s and revised in the 1990’ s, depicts six stages 
of student information activity: initiation, selection, exploration, formulation, collection, and presentation 
(Kuhlthau, 1991). These six stages reflect the primary tasks to be accomplished when students go beyond 
social connnunications technology exchanges and engage in complex information behaviors that can be 
useful for directing learning activities. Emphasis should be placed on supporting information seeking tasks 
which are associated with independent learning and knowledge construction (George, Bright, Hurlbert, 
Linke, St. Clair & Stein, 2006). Established for an understanding of student information behavior in support 
of guided instructional, diagnostic, and intervention programs, Kuhlthau’s ISP model has been applied 
extensively for decades. An extensive review of literature and inquiry project conducted in 2007 among 
(n=574) school students indicated that Kuhlthau’s information seeking model for affective, cognitive, and 
physical dimensions of information behavior continues to be usefulness for explaining student information 
seeking behavior and knowledge acquisition in the digital, technology pervasive information environment of 
Web 2.0 (Kuhlthau, Heinstrom, & Todd, 2008). Findings confirm that while some stages in the search 
process may be intensified in new instantaneous information environments “the Information Search Process 
seems to be an over-arching process regardless of search venue, or print or digital format” (Kuhlthau, et al, 
2008). 

Mobile learning is a relatively new phenomenon with its theoretical basis still under development 
(Kearney, Schuck, Burden, Aubusson, 2012). With the rapid growth of mobile devices throughout the world 
(World Bank, 2012), the need has emerged for studies of affordances and barriers that might enhance or 
constrain the adoption of mobile learning in higher education. This approach is consistent with the principles 
of diffusion of innovations (Rogers, 2003) that have served as a useful model for studying the introduction of 
new information technologies in the past, and which have been suggested by Chueng and Hew (2009) as an 
appropriate framework for examining the adoption of mobile devices in teaching and learning. 

While mobile learning is recognized as the desired outcome for application of mobile technology to 
distance learning, the widespread availability of mobile technology and the association of mobile learning 
with new forms of communications related activities for engaged learning spotlights mobile learning for a 
broad range of formal and informal learning activities in the twenty-first century (Andrews and Tynan, 
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2012). Cox (2012) contends that research approaches investigating innovative ways of teaching and learning 
with ICT in the future should address technology-enhanced learning “outside formal educational settings” (p. 
2) as well as the opportunities presented by “the uptake of thin client technologies” (p. 5) that are mobile and 
personalized. Lai (2011) devoted a section of his work on digital technology and the culture of teaching and 
learning in higher education to “... how digital technologies may provide a more active and flexible learning 
experience by adopting a participatory pedagogical approach and by blending formal learning with informal 
learning” (p. 1263). 


3. DEVELOPMENT OF THE INFORMATION AND COMMUNICATIONS 
TECHNOLOGY LEARNING (ICTL) SURVEY 

The Information and Communications Technology Learning (ICTL) survey was designed and validated to 
help address questions related to how students prefer to utilize ICT for information seeking, information 
sharing, and knowledge acquisition. Instrument development included analysis for internal consistency 
reliability, principal components exploratory factor analysis, multidimensional scaling, and higher order 
factor analysis. Items were initially gathered by the first author in an effort to better understand learner 
preferences and the role that ICT plays in learning today. A review of literature did not reveal validated 
instruments for measurement of student learning preference for use of ICT for learning activities and 
knowledge acquisition within the ubiquitous communications landscape of the twenty-first century (Mills and 
Knezek, 2012). Items for a 15 -item prototype instrument, the Information Communications and Technology 
Learning (ICTL) survey version 1 .0, are displayed in Figure 1 . Survey questions are Likert-type, rated on a 5 
point scale with response choices from 1= strongly disagree to 5 = strongly agree. Note that item 9 is 
negatively worded requiring that response ratings be reversed before sunnning (or averaging) with ratings 
from other items to form a Likert- type scale. The reverse coding for item 9 results in answer choices having 
increasing value along the response scale 1-5. 


Information and Communications Technology Learning (ICTL) survey 

1 . 1 would like to be a participating member of an online community. 

2. 1 use Internet technology to explore topics of interest. 

3. 1 like to share interests and reflections online. 

4. 1 like to enroll in classes to continue my education. 

5. 1 use Internet communications and other technology tools for self-expression. 

6. 1 learn many things by interacting with other Internet users. 

7. 1 like to take classes from good professors. 

8. 1 use Internet communications technology tools when I want to learn about something new. 

9. 1 learn best in a traditional classroom setting. (R) 

10. Internet technology helps me be successful in my college classes. 

11. More classroom learning should include interactive communication technology experiences. 

12. The things I need to know are taught by instructors in the classroom. 

13.1 learn more when I regulate my own learning experience and seek information on things that I want to 
learn about. 

14. 1 use Internet communications technology to keep current on topics related to my field of expertise. 

15.1 post information that might be of interest to other people. 

Eigure 1. Information and Communications Technology Learning (ICTL) survey items. Note: ICTL vl.O 2011 by L. 

Mills & G. Knezek. 


4. DATA COLLECTION 

Survey subjects were volunteer participant college students enrolled in one of two institutions of higher 
education in Texas (USA), a junior college with a two-year curriculum, and a four-year state university. 
Sixty-two (62) respondents completed the ICTL survey online during the fall semester of 2011. The survey 
participants were 89% women (n=55) and 11% men (n=7) spanning 18 to 59 years of age. 
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5. INSTRUMENT REFINEMENT 


5.1 Internal Consistency Reliability and Construct Validity 

Reliability analysis was used to assess the internal consistency of the instrument scales according to 
Cronbach’s Alpha index. As shown in Table 1, Cronbach’s Alpha for all 15 items of the ICTL instrument is 
considered “respectable”, (alpha = .77) according to reliability guidelines by DeVellis (1991). 

Exploratory factor analysis, principal components analysis (PC A) with varimax rotation, was conducted 
in order to identify scales/factors that are orthogonally aligned (Mertler & Vannatta, 2005). PCA produced 
four factors with eigenvalues greater than one. Scree plot analysis (see Figure 2) indicated a two or possibly 
four factor solution in light of Stevens’ (1992) suggestion that constructs in the sharp decent of the graph, 
before the first point of leveling, be retained. The four-factor solution was selected because the items within 
each factor were judged to have greater content validity. These four factors (see Table 1) were tentatively 
named Online Reflection, Internet Exploration, ICT Research, and Classroom Learning, with alphas of 0.88, 
0.70, 0.43, and 0.54, respectively, when subjected to post hoc internal consistency reliability analysis. 
Measurement scales produced for these factors were found to have internal consistency reliabilities ranging 
from very good for Online Reflection to unacceptable for Classroom Learning, according to guidelines by 
DeVellis (1991). 


Table 1. Cronbach’s Alpha for ICTL. Internal consistency reliabilities for ICTL scales 


Scale 

No. of 

Item 

Cronbach’s Alpha 

DeVellis 


Items 

numbers 


Guidelines 

ICTL- Total scale 

15 

1-15 

0.77 

Respectable 

Online Reflection 

6 

3,5, 11,6, 
15, 1 

0.88 

Very Good 

Internet 

Exploration 

4 

7, 2, 13,4 

0.70 

Respectable 

ICT Research 

3 

8, 10, 14 

0.43 

Unacceptable 

Classroom Learning 

2 

12,9 

0.54 

Unacceptable 


Scree Plot 



Figure 2. Scree plot for exploratory factor analysis of ICTL items 

Higher order factor analysis was conducted to explore explanatory constructs at a higher level connnon 
among the four factor scales identified by PCA: Online Reflection, Internet Exploration, ICT Research, 
Classroom Learning (see Table 2). Scale scores produced for each of the factors were used as input variable 
values for the higher order procedure. A Scree plot for higher order factor analysis (Figure 3) reveals two 
factors in the sharp decent of the line. Table two shows that these two components explain 70% of the total 
variance. 
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Table 2. Higher order factor analysis loadings for four ICTL scales. Two components account for 70% of variance 
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Figure 3. Scree plot for higher order factor analysis 

Internal consistency reliability was computed for these two higher-order construct scales. The first scale, 
Information Seeking was found to have alpha = .71 (respectable according to DeVellis (1991)). Cronbach’s 
Alpha index for the second scale, Information Sharing was .83 (very good per DeVellis’ guidelines (1991)). 
The items forming each higher-order scale are listed in Table 3. 

5.2 Higher Order Factor Analysis and Multidimensional Scaling 

Higher order factor analysis and multidimensional scaling indicated two possible factors which, upon 
examination by researchers, proved to have content (face) validity. One factor consisted of items that were 
related to ICT use for reflection, connnunication, and sharing. The second factor grouped items related to 
ICT for seeking information related to areas of interest, expertise, or study. Multidimensional scaling was 
conducted to further examine the underlying factors for the ICTL survey. The ALSCAL Euclidian distance 
model with maximum of two dimensions was generated to examine distances and proximities for items in 
relation to one another. Two main output clusters are visible on the Euclidean distance model (Eigure 4). This 
two-scale alignment confirms results of higher order factor analysis. Instrument items in quadrant II & III, 
with item #8, which is located near the Y axis were examined as the first of two scales, and all other items in 
quadrants I & IV as the second of two scales which were identified by use of higher order factor analysis and 
multidimensional scaling. This two-factor solution was accepted and resulted in the Information Seeking, and 
Information Sharing scales of the ICTL. 
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Derived Stimulus Configuration 
Euclidean distance model 



Figure 4. Euclidean distance model. Multidimensional scaling plot of ICTL items 


Table 3. Items for higher order scales. Two scales emerged from higher order factor analysis 


ICTL Scale 

# Items 

Item 

Numbers 

Alpha 

Cronbach’s 

Rating (DeVellis) 

ICTL- Total scale 

15 

1-15 

0.77 

Respectable 

subscales: 

Information Seeking 

7 

2,4,7,8,10,13, 

14 

0.71 

Respeetable 

Information Sharing 

8 

l,3,5,6,9r,ll, 

12,15 

0.83 

Very Good 


Note: Item #9 was reverse coded. 


5.3 Criterion-related Validity 

Criterion related validity was examined for the three scales of ICTL by correlation analysis with the Mobile 
Learning Scale, a 7-item, unidimensional, Likert- type survey instrument developed by Khaddage and Knezek 
(2011) for used with higher education students. The Mobile Learning Scale version 1.0 was designed to 
assess acceptance of mobile learning, including perceptions of mobile learning devices and tools (Apps) for 
informal learning, and feelings about using theories and models to incorporate mobile learning into higher 
education (Knezek & Khaddage, 2011). Knezek and Khaddage (2011) found the internal consistency 
reliability for the seven-item instrument to be very good (alpha = 0.85) (DeVellis, 1991) among 81 
undergraduate and graduate university students completing the survey in a large Midwestern university in the 
USA during August and September of 2011. A list of the items on the Mobile Learning Scale version 1.0 is 
provided in Figure 5. Correlation analysis revealed concurrent validity between ICTL, Information Seeking, 
Information Sharing and the Mobile Learning Scale. Significant correlations (p < .01) were found between 
Mobile Learning (ML) total scale score and all three ICTL measures: a) ML with ICTL_Total r = .37 (p < 
.0005), b) ML with ICTL_Information Seeking r =.35 (p < .0005), and c) ML with ICTLJnformation 
Sharing r =.37 (p < .0005). This magnitude of Pearson Product Moment Correlation would be considered a 
moderate effect size according to guidelines by Cohen (1988). This confirms that the ICTL survey scales 
have demonstrable criterion related validity in the form of alignment with an established Mobile Learning 
Scale. This research finding contributes new knowledge to the emerging set of models and theories 
developing in the field of mobile learning. In particular, the implication is that both information seeking and 
information sharing are important aspects to be considered in mobile learning environments 
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Instructions: Select one level of agreement for each statement to indicate how you feel. 


SD = Strongly Disagree, D = Disagree, U = Undecided, A = Agree, SA = Strongly Agree 



MOBILE LEARNING SCALE 

SD 

D 

u 

A 

SA 

1. 

The rapid development of Mobile Learning devices and tools 
(Apps) has empowered informal learning. 

® 

(2: 

(3 

A} 

© 

2. 

Mobile Apps could be integrated seamlessly to support informal 
learning. 

® 

(V 

(3: 

© 

© 

3. 

Mobile Apps could bring enormous opportunities into universities 
to further empower informal learning 

® 

(2: 

(T 


© 

4. 

Student acceptance of Mobile Learning in higher education would 
be high. 

© 


(3: 


© 

5. 

Recent developments in Mobile Learning are leading to the 
exploration of new methods/models at universities 

© 

(2: 

(3: 

Gj 

© 

6. 

Theoretical models and methods can assist in informing the design 
for mobile learning Apps 

© 

(2.1 

(3: 

© 

©' 

7. 

The integration of mobile applications, mobile social networking 
platforms and other mobile technologies has become pervasive in 
teaching and learning. 

© 

(V 

(3: 

© 

©■ 


Figure 5. Mobile Learning Scale (MLS) Version 1.0 items. Note: MLS vl.O 8/2011 by F. Khaddage & G. Knezek 


Additional associations were found between other attributes of interest and one or more of the scales used 
in this study. Females tended to rate higher than males on mobile learning attitudes (p = .012) but there were 
no significant differences by gender on information seeking or information sharing preferences. Age was 
negatively correlated with information seeking ratings (p = .008) and information sharing (p = .001), but no 
significant correlation was found between age and mobile learning attitude (NS). Mobile learning attitudes 
were positively correlated (p = .002) with self-reported creative tendencies as measured by a 13 item Likert 
Scale. These findings add context to the associations reported in the previous paragraph, and raise additional 
questions to be addressed in future research. 


6. DISCUSSION 

In recent decades the role of formal education as the disseminator of significant learning experiences has 
diminished (Aittola, 1999). Information communications technologies of the twenty-first century support a 
variety of learning, sharing and knowledge acquisition options which can take place in many places 
facilitated by internet-based communications tools (Knezek, Lai, Khaddage & Baker, 2011), thereby altering 
the role of formal education in learning. The information seeking, sharing and mobile access dimensions of 
learning which are measured by validated instruments identified in this paper are well fitted to assist teachers 
and learning technologists to design instruction that will support learning interaction with new options in 
mobile learning technologies and ICT. 

As stated by Naismith, Lonsdale, Vavoula, and Sharpies, (2004, p. 36) the current challenge is for 
educators and technology developers to find a way to ensure that new learning methods are effective - highly 
situated, personal and collaborative for the long term, and accepted by students. Research for formal and 
informal learning in new technology pervasive environments should include data on student preferences that 
can help identify students’ attitudes, level of acceptance, opinions and expectations regarding the integration 
of new technologies such as mobile learning into teaching and learning. This approach is what Andrews and 
Tynan (2012) have referred to as “investigating the human voice” (p. 565) to meet the unique needs of 
today’s learner. Hence, by creating and validating instruments to measure attitudes toward adoption and use 
of new technology options, such as the ones presented in this paper, which can more precisely and efficiently 
assess reliability and validity and make continuous refinements, we can allow for informed understanding of 
ICT learning and information seeking preferences of students. Additionally, the findings of this study imply 
that the ICTL and Mobile Learning Scale are useful for measurement in three areas of focus for the study of 
students’ needs: information seeking preferences and behaviors, information sharing preferences and 
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behaviors, and perceptions of teaching and learning with mobile devices. 

A review of the literature revealed that existing instruments for measurement of technology in teaching 
and learning have frequently focused on self-reported measures that assess student and teacher attitudes and 
proficiency with information and communication technology for technology integration (Knezek & 
Christensen, 2000). Instruments have also been validated to measure success with teaching and learning in 
Internet enriched learning environments (Hong, Ridzuan, & Kuek, 2003). A number of instruments measure 
different facets of learning. For example, Litzinger, Wise, Lee, & Bjorklund (2003), developed an instrument 
to measure aspects of self-directed learning. The authors contend that there is a need for the ICT and mobile 
learning scales to measure learner preference for ICT and mobile technologies for information behavior such 
as seeking and sharing. 


7. SUMMARY AND CONCLUSIONS 

Data collected online from college student volunteers for a learning preference study was used to refine the 
Information Communication Technology Learning (ICTL) survey. Construct validity was examined in a 
multi-step process. Reliability was analyzed for all 15 items of this survey and found to be respectable (alpha 
= .77) according to established guidelines (DeVellis, 1991). Four possible measurement factors were 
identified by PC A exploratory factor analysis: two of these were considered unacceptable with low internal 
consistency reliabilities according to established guidelines. However, higher order factor analysis and 
multidimensional scaling identified a two higher order constructs with very good and respectable 
measurement properties: Information Sharing (alpha = .83) and Information Seeking (alpha = .71). Scales 
produced from these two factors were found to be significantly correlated (p < .0005) with student mobile 
learning (MLS) attitudes, providing cross-validation evidence for ICTL scales. The correlations between 
ICTL scales and MLS were at the r = .35 level or higher, indicating that sharing and seeking information both 
are perceived as functions to be performed with mobile technologies. These findings support the authors’ 
conclusion that the Information and Communications Technology Learning (ICTL) instrument is worthy of 
further use, either alone or as part of a battery of instruments including others such as the Mobile Learning 
Scale. 

This study also contributes new knowledge to the emerging set of models and theories developing in the 
field of mobile learning. In particular, the implication is that both information seeking and sharing behaviors 
are important points of consideration in mobile learning environments. Further research is needed to 
determine if the instruments used for this research can assist educators and researchers in better 
understanding student preferences for ICT integration in order to improve support for information behavior 
targeting formal and informal learning. Findings from this study, such as the positive correlations identified 
between mobile learning attitudes and creative tendencies, and the confirmed female preference for mobile 
learning while no gender differences regarding information seeking or sharing were found, suggest that the 
ICTL and MLS instruments possess criterion related characteristics that may be useful in addressing 
additional research questions on students’ attitudes towards mobile devices, ICT integration, and information 
seeking behaviors. These areas will be pursued in future studies. 
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